Stereoscopic image display device and eye-tracking method thereof

ABSTRACT

Discussed are a stereoscopic image display device to implement high-speed eye-tracking techniques and an eye-tracking method. The stereoscopic image display in one embodiment includes an image panel to alternately display left-eye and right-eye images, a switchable panel disposed at the front or rear side of the image panel to separate the left-eye and the right-eye images from each other to correspond to the left and right eyes of a viewer, a camera mounted to the image panel to capture an image of the viewer, and a computer system to detect position information of the viewer from the image input and calculate midpoint information between the detected position information by interpolation using the detected position information to update position information of the viewer at a faster drive frequency than a drive frequency of the camera, and to control driving of the switchable panel in response to the updated position information.

This application claims the benefit of the Korean Patent Application No.10-2013-0038815, filed on Apr. 9, 2013 in Republic of Korea, which ishereby incorporated by reference as if fully set forth herein.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a stereoscopic image display device,and more particularly, to a stereoscopic image display device, which mayimplement high-speed eye-tracking techniques using a low-speed camera,and an eye-tracking method thereof.

Discussion of the Related Art

Stereoscopic image display devices, designed to display a realistic3-dimensional (3D) image, have been applied to a variety of fieldsincluding medicine, education, games, movies, televisions, and the like.Such a stereoscopic image display device displays a left-eye image and aright-eye image separate from each other in terms of space or time tocause a viewer to perceive 3D effects from binocular parallax images.

Representative methods to display a 3D image include a glasses methodusing special glasses and a non-glasses method not using specialglasses. In the glasses method, a display device displays a left-eyeimage and a right-eye image by changing polarization of the images or byseparating the images from each other in a time division manner, and aviewer perceives 3D effects using polarized glasses or liquid-crystalshutter glasses. In the non-glasses method, a display device displays a3D image using optical filters installed to a front or rear facethereof, such as a lenticular sheet, a parallax barrier, and the like.

More specifically, a non-glasses type stereoscopic image display deviceincludes an image panel to display an image, and a switchable paneldisposed at the front or rear side of the image panel to spatiallyseparate a left-eye image and a right-eye image from each other byvarying a barrier position or a lens position per the left-eye image andthe right-eye image.

In addition, the stereoscopic image display device utilizes eye-trackingtechniques, which detects the face of a viewer from an image input via acamera mounted to the image panel, and subsequently detects eye positioninformation of the viewer relative to the stereoscopic image displaydevice (e.g., X, Y, and Z coordinates), and display a 3D image whiletracking a position of the viewer who is moving by varying a barrierposition or a lens position of the switchable panel based on thedetected eye position information of the viewer.

Conventionally, the stereoscopic image display device synchronizesupdate of position information of the viewer with a drive frequency(image capture speed) of the camera. However, the camera is driven at alower speed than a drive frequency of the switchable panel, and thusposition information of the viewer is updated at a lower speed than thedrive frequency of the switchable panel, which makes it impossible todisplay a 3D image accurately conforming to a position of the viewer whois moving. Although this problem may be solved by using a camera havinga high drive frequency, a low-speed camera is generally used because ofrestrictions in terms of price and size of the camera.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a stereoscopic imagedisplay device and an eye-tracking method thereof that substantiallyobviate one or more problems due to limitations and disadvantages of therelated art.

An object of the present invention is to provide a stereoscopic imagedisplay device, which may implement high-speed eye-tracking techniquesusing a low-speed camera, and an eye-tracking method thereof.

Additional advantages, objects, and features of the invention will beset forth in part in the description which follows and in part willbecome apparent to those having ordinary skill in the art uponexamination of the following or may be learned from practice of theinvention. The objectives and other advantages of the invention may berealized and attained by the structure particularly pointed out in thewritten description and claims hereof as well as the appended drawings.

To achieve these objects and other advantages and in accordance with thepurpose of the invention, as embodied and broadly described herein, astereoscopic image display device includes an image panel configured toalternately display a left-eye image and a right-eye image, a switchablepanel disposed at the front or rear side of the image panel to separatethe left-eye image and the right-eye image from each other so as tocorrespond to the left eye and the right eye of a viewer, a cameramounted to the image panel to capture an image of the viewer, and acomputer system configured to detect position information of the viewerfrom the image input by the camera and calculate midpoint information ofthe viewer between the detected position information by interpolationusing the detected position information to update position informationof the viewer at a faster drive frequency than a drive frequency of thecamera, and also configured to control driving of the switchable panelin response to the updated position information of the viewer.

The computer system may detect the face of the viewer from the imageinput by the camera, and then may detect two pieces of eye positioninformation of the viewer from the detected face, and may calculate npieces of midpoint information (here, n is a natural number) between thetwo pieces of eye position information via interpolation using thedetected two pieces of eye position information. Each piece of the eyeposition information of the viewer may include X, Y, and Z coordinates,and the computer system may calculate the n pieces of midpointinformation by summing the two pieces of eye position information percoordinate and dividing the sum per coordinate by 1/n.

The computer system may update the eye position information of theviewer at 2× or 4× the drive frequency of the camera.

In accordance with another aspect of the invention, an eye-trackingmethod of a stereoscopic image display device, includes inputting animage of a viewer, captured by a camera mounted to an image panel,detecting the face of the viewer from the input image and detecting eyeposition information of the viewer from the detected face, calculatingmidpoint information of the viewer between the detected eye positioninformation of the viewer by interpolation using the detected eyeposition information, to update the eye position information of theviewer at a faster drive frequency than a drive frequency of the camera,and controlling driving of a switchable panel disposed at the front orrear side of the image panel in response to the updated positioninformation of the viewer.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 is a block diagram schematically showing a configuration of astereoscopic image display device according to an embodiment of thepresent invention;

FIG. 2 is a flowchart showing the sequence of an eye-tracking method ofa computer system shown in FIG. 1;

FIG. 3 is a diagrammatic view showing an eye-tracking method of astereoscopic image display device according to an embodiment of thepresent invention; and

FIG. 4 is a view showing position information of a viewer updated by theeye-tracking method of the stereoscopic image display device accordingto the embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings.

FIG. 1 is a block diagram schematically showing a configuration of astereoscopic image display device according to an embodiment of thepresent invention.

The stereoscopic image display device, as exemplarily shown in FIG. 1,includes a dual-panel type image panel 10 to display a left-eye imageand a right-eye image, and a switchable panel 20 disposed at the frontor rear side of the image panel 10 to separate the left-eye image andthe right-eye image from each other by varying a position of a barrierthat shields light or a position of a lens that refracts light based onthe left-eye image and the right-eye image of the image panel 10 andposition information of a viewer. For convenience of description,although FIG. 1 shows only a configuration in which the switchable panel20 is disposed at the front side of the image panel 10, when the imagepanel 10 is a liquid crystal panel using a backlight unit as a lightsource, the switchable panel 20 may be disposed at the rear side of theimage panel 10, i.e. between the backlight unit and the image panel 10.Hereinafter, only the configuration in which the switchable panel 20 isdisposed at the front side of the image panel 10 will be described byway of example.

The image panel 10 alternately displays a left-eye image and a right-eyeimage in a time division manner for display of a 3D image. For example,the image panel 10 displays a left-eye image at an odd frame anddisplays a right-eye image at an even frame to alternately display theleft-eye image and the right-eye image per frame. Alternatively, theimage panel 10 may display a 2D image.

The image panel 10 may be a liquid crystal panel, an organic lightemitting diode display panel, a plasma display panel, or the like. Inone example in which the image panel 10 is a liquid crystal panel, theimage panel 10 includes a color filter substrate having a color filterarray, a thin film transistor substrate having a thin film transistorarray, a liquid crystal layer encapsulated between the color filtersubstrate and the thin film transistor substrate, and polarizer platesattached respectively to outer surfaces of the color filter substrateand the thin film transistor substrate. The image panel 10 displays animage via a matrix of multiple pixels. Each pixel emits a desired colorof light using a combination of red/green/blue (R/G/B) sub-pixels whichadjust light transmittance as arrangement of liquid crystals variesbased on a data signal, and further includes a white (W) sub-pixel toenhance brightness as needed. Each sub-pixel includes a thin filmtransistor (TFT) connected to a gate line (GL) and a data line (DL), anda liquid crystal capacitor (Clc) and a storage capacitor (Cst) connectedto the thin film transistor in parallel. The liquid crystal capacitor ischarged with a differential voltage between a data signal applied to apixel electrode through the thin film transistor and a common voltage(Vcom) applied to a common electrode, and adjusts light transmittance bydriving liquid crystals based on the charged voltage. The storagecapacitor stores the charged voltage of the liquid crystal capacitor.The liquid crystal layer is driven by a vertical field as in a TwistedNematic (TN) mode or a Vertical Alignment (VA) mode, or is driven by ahorizontal field as in an In-Plane Switching (IPS) mode or in a FringeField Switching (FFS) mode.

The switchable panel 20 is attached to a front face of the image panel10 and includes a liquid crystal panel, which serves as an opticalfilter, such as a switchable barrier to vary a position of a barrier ora switchable lens to vary a position of a lens based on the left-eyeimage and the right-eye image of the image panel 10 and positioninformation of the viewer when the image panel 10 displays a 3D image.Alternatively, when the image panel 10 is the liquid crystal panel, theswitchable panel 20 may be disposed between the backlight unit and theimage panel 10.

When the image panel 10 displays a 3D image, the switchable panel 20enters a barrier mode or a lens mode such that the left-eye image andthe right-eye image displayed on the image panel 10 are separatelyperceived by the left eye and the right eye of the viewer to allow theviewer to view a 3D image. Alternatively, when the image panel 10displays a 2D image, the switchable panel 20 exits the barrier mode orthe lens mode such that all cells are driven in the same lighttransmitting mode (2D mode) to allow the viewer to view the 2D imagedisplayed on the image panel 10.

Hereinafter, only the case in which the image panel 10 and theswitchable panel 20 display a 3D image will be described.

More specifically, when the image panel 10 displays a left-eye image,the switchable panel 20 varies a barrier position or a lens position todisplay the left-eye image at a position of the left eye of the viewer.In addition, when the image panel 10 displays a right-eye image, theswitchable panel 20 varies a barrier position or a lens position basedon position information of the viewer to display the right-eye image ata position of the right eye of the viewer. In this way, the left-eyeimage and the right-eye image, sequentially displayed on the image panel10, are separately perceived by the left-eye and the right-eye of theviewer, which allows the viewer to view a 3D image.

The switchable panel 20 described above is a mono panel in which acorresponding cell is turned on or off based on a drive voltage toselectively shield light, or in which an index of refraction of liquidcrystals is variable based on a drive voltage to form a lens.

The stereoscopic image display device, as exemplarily shown in FIG. 1,further includes a first panel drive unit 12 to drive the image panel10, a second panel drive unit 22 to drive the switchable panel 20, acomputer system 30 connected to the first and second panel drive units12, 22, and a camera 40 to transmit a captured image of the viewer tothe computer system 30.

The first panel drive unit 12 drives the image panel 10 such thatleft-eye image data and right-eye image data fed from the computersystem 30 are displayed on the image panel 10. To this end, the firstpanel drive unit 12 may include a first data driver (not shown) to drivedata lines of the image panel 10, a first gate driver (not shown) todrive gate lines, and a first timing controller (not shown) to controldrive timing of the first data driver and the first gate driver.

The second panel drive unit 22 drives the switchable panel 20 usingcontrol data fed from the computer system 30 based on left-eye andright-eye images and position information of the viewer. The secondpanel drive unit 22 may include a second data driver (not shown) and asecond gate driver (not shown) to drive data lines and gate lines of theswitchable panel 20 respectively, and a second timing controller (notshown) to control drive timing of the second data driver and the secondgate driver.

The camera 40 traces a position of the viewer and transmits a capturedimage of the viewer to the computer system 30.

The computer system 30 feeds the left-eye image data and the right-eyeimage data, which will be displayed on the image panel 10, and multiplesynchronization signals to the first panel drive unit 12. In addition,the computer system 30 detects position information of the viewer fromthe image input via the camera 40, produces control data to controldriving of the switchable panel 20 based on the detected positioninformation, and feeds the same to the second panel drive unit 22. Inthis case, the computer system 30 may update position information of theviewer at a faster drive frequency than a drive frequency of thelow-speed camera 40 by detecting more than one midpoint betweenproximate first and second positions of the viewer via interpolationusing first and second position information of the viewer detected fromthe image of the camera 40.

More specifically, an eye-tracking algorithm of the computer system 30as exemplarily shown in FIG. 2 is as follows.

In a step S2, the computer system 30 inputs an image captured by thecamera 40, and in Operation S4, the computer system 30 detects the faceof the viewer from the input image. For example, the computer system 30detects the face of the viewer from the input image via face detectionusing, e.g., a Haar classifier.

In a step S6, the computer system 30 detects coordinates of the centerbetween the left eye and the right eye (hereinafter referred to as eyecenter) from the detected face of the viewer. For example, the computersystem 30 initially selects specific points, such as the left eye andthe right eye, using an eye model, such as, e.g., an Active AppearanceModel (AAM), and then detects coordinates of the eye center as a finalspecific point using, e.g., an Elastic Bunch Graph Matching (EBGM)model.

In a step S8, the computer system 30 calculates eye position informationof the viewer by applying the detected coordinates of the eye center toan eye-based distance model. The eye position information of the viewer,as exemplarily shown in FIG. 3, includes X, Y, and Z coordinatescalculated on the basis of the center of the stereoscopic image displaydevice. In this case, the computer system 30 detects eye positioninformation of the viewer from an input image whenever the input imageis input from the camera 40, the eye position information of the vieweris detected at the same drive frequency as a drive frequency of thecamera 40.

In a step S10, the computer system 30 calculates more than one midpointbetween proximate first and second eye positions by interpolation usingdetected first and second eye position information P1, P2 to update theposition information of the viewer at a faster drive speed than thedrive frequency of the camera 40. For example, the computer system 30may detect eye position information of the viewer at a drive frequencyof 30 Hz from the image input by the camera 40, and calculate midpointinformation between the detected eye position information byinterpolation using the detected eye position information to update theposition information of the viewer at a frequency of 60 Hz or 120 Hz.

When attempting to calculate n pieces of midpoint information betweenthe first and second eye position information P1, P2 (n being a naturalnumber), the n pieces of midpoint information may be calculated bydividing the sum of respective coordinates as the first and second eyeposition information P1, P2 by 1/n.

For example, as exemplarily shown in FIG. 4, the computer system 30 maydetect first eye position information X1, Y1, Z1 and second eye positioninformation X2, Y2, Z2 of the viewer from the image input by the camera40. Then, when it is intended to calculate three pieces of midpointinformation between the detected first eye position information X1, Y1,Z1 and the detected second eye position information X2, Y2, Z2, thecomputer system 30 may calculate the sum of the detected first eyeposition information X1, Y1, Z1 and the detected second eye positioninformation X2, Y2, Z2 per each coordinate, and divide the sum of therespective coordinates by ⅓. In this way, the computer system 30 mayupdate eye position information of the viewer at 4× the drive frequencyof the image input from the camera 40.

In a step S12, the computer system 30 produces control data to drive theswitchable panel 20 based on the updated position information of theviewer, and feeds the same to the second panel drive unit 22.

As is apparent from the above description, in a stereoscopic imagedisplay device and an eye-tracking method thereof according to thepresent invention, first position information and second positioninformation of a viewer are detected from an image captured by alow-speed camera, and midpoint information between the first and secondposition information of the viewer is calculated via interpolation usingthe detected first and second position information, which enables updateof position information of the viewer at a speed similar to a drivefrequency of a high-speed driven switchable panel. This faster update ofposition information of the viewer than a drive speed of the low-speedcamera may attain more accurate tracking of a position of the viewer whois moving than in the related art, resulting in formation of a morenatural 3D image than that of the related art.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present inventionwithout departing from the spirit or scope of the inventions. Thus, itis intended that the present invention covers the modifications andvariations of this invention provided they come within the scope of theappended claims and their equivalents.

What is claimed is:
 1. A stereoscopic image display device comprising:an image panel configured to alternately display a left-eye image and aright-eye image; a switchable panel disposed at a front or rear side ofthe image panel to separate the left-eye image and the right-eye imagefrom each other so as to correspond to the left eye and the right eye ofa viewer; a camera mounted to the image panel to capture an image of theviewer; and a computer system configured to: provide left-eye andright-eye image data to a first panel drive unit to display the left-eyeand right-eye images on the image panel, detect a face of the viewerfrom the captured image of the viewer, detect eye position informationincluding first eye position information at a first position and secondeye position information at a second position of the viewer from thedetected face, each of the first and second eye position informationincluding x, y, and z coordinates, calculate more than one midpointinformation of the viewer between the detected first eye positioninformation and second eye position information by interpolation usingthe detected first and second eye position information, wherein thecomputer system detects the first and second eye position information ofthe viewer when the captured image of the viewer is input from thecamera at a first drive frequency, and calculates the more than onemidpoint information between the detected first and second eye positioninformation to update the eye position information of the viewer at asecond drive frequency higher than the first drive frequency, producecontrol data to control driving of the switchable panel based on theupdated eye position information of the viewer, and provide the controldata to a second panel drive unit to drive the switchable panel at adrive frequency higher than the first drive frequency of the camera,wherein the computer system detects first center coordinates between theleft eye and right eye at the first position of the viewer to detect thefirst eye position information and second center coordinates between theleft eye and the right eye at the second position of the viewer todetect the second eye position information using the images input by thecamera, and calculates center coordinates of the more than one midpointposition between the first and second positions of the viewer to updatethe eye position information.
 2. The device according to claim 1,wherein the computer system detects faces of the viewer from the imagesinput by the camera, and then detects the first center coordinates andthe second center coordinates from the detected faces.
 3. The deviceaccording to claim 2, wherein the computer system updates the first andsecond eye position information as the center coordinates of the viewerat 2× or 4× the drive frequency of the camera.
 4. A method of operatinga stereoscopic image display device, the method comprising: providingleft-eye and right-eye image data to a first panel drive unit toalternately display left-eye and right-eye images via an image panel;separating the left-eye and right-eye images from each other via aswitchable panel disposed at a front or rear side of the image panel soas to correspond to the left and right eyes of a viewer; receivingimages of the viewer, captured by a camera mounted to the image panel;detecting, via the camera, a face of the viewer from the captured imagesof the viewer and detecting, via a computer system, eye positioninformation including first eye position information at a first positionand second eye position information at a second position of the viewerfrom the detected face, each of the first and second eye positioninformation including x, y, and z coordinates; calculating, via thecomputer system, more than one midpoint information of the viewerbetween the detected first eye position information and second eyeposition information by interpolation using the detected eye positioninformation, wherein the computer system detects the first and secondeye position information of the viewer when the captured image of theviewer is input from the camera at a first drive frequency, andcalculates the more than one midpoint information between the detectedfirst and second eye position information to update the eye positioninformation of the viewer at a second drive frequency higher than thefirst drive frequency; producing control data to control driving of theswitchable panel based on the updated eye position information of theviewer; and providing the control data to a second panel drive unit todrive the switchable panel at a drive frequency higher than the firstdrive frequency of the camera, wherein the detecting the eye positioninformation of the viewer comprises: detecting first center coordinatesbetween the left-eye and right-eye at the first position of the viewerto detect the first eye position information and second centercoordinates between the left eye and the right eye at the secondposition of the viewer to detect the second eye position informationusing the images input by the camera; and calculating center coordinatesof the more than one midpoint position between the first and secondpositions of the viewer to update the eye position.
 5. The methodaccording to claim 4, wherein the first and second eye positioninformation as the center coordinates of the viewer is updated at 2× or4× the drive frequency of the camera.